1. Field of the Invention
This invention relates to the field of video processing, and in particular to processes used to enhance a video image.
2. Description of Related Art
The xe2x80x9csharpnessxe2x80x9d of an image can be improved by enhancing the high-frequency components of the image; that is, by accentuating edges and other portions of an image so that changes are sharp, rather that gradual. A high-frequency enhancement, however, can cause some changes to be overly accentuated, resulting in a xe2x80x9cspecklingxe2x80x9d of the image and other visually unappealing anomalies.
FIG. 1 illustrates a block diagram of a conventional sharpness enhancement device 100. The sharpness enhancement device 100 includes a convolution kernel 150 that functions as a high frequency filter to identify the high frequency components of the picture. Conventionally, an image on a television screen is commonly referred to as a picture. The terms image and picture are used interchangeably herein, and are meant to include all or parts of a set of data that can be processed to produce a visual rendering corresponding to the data. The convolution kernel 150 computes a convolution value for each pixel of an image, based on the pixel""s visual characteristics compared to the characteristics of neighboring pixels. A convolution kernel that is commonly used to provide this value is:       [                            0                                                    -              1                        /            4                                    0                                                                -              1                        /            4                                                +            1                                                              -              1                        /            4                                                0                                                    -              1                        /            4                                    0                      ]    ,
the center of this kernel corresponding to the pixel being processed. In this example, the value of the pixel being processed is multiplied by +1, and the values of the pixels immediately above, below, left, and right of the pixel being processed are each multiplied by xe2x88x92xc2xc, and the sum of these multiplied values is the determined convolution value for the pixel being processed. If, for example, the pixel is located in a region of uniform pixel values, the sum of the pixel value minus a quarter of each of four similar pixel values is zero. That is, no enhancements are made within a region of uniform pixel values. Conversely, if the pixel value is 100, and it is surrounded by pixel values of 40 each; the convolution value is 60 (100xe2x88x9210xe2x88x9210xe2x88x9210xe2x88x9210). That is, the larger the change of a pixel""s value, relative to neighboring pixels, the larger the convolution value. This convolution value, C, is appropriately scaled by a gain factor, g, at 170, and added to the original pixel value, Yin, at 180, to form a sharpness enhanced pixel value:
Yout=Yin+g*C.xe2x80x83xe2x80x83(1)
The determination of an appropriate gain factor, g, in this example includes four processes 110-140; less costly systems may use fewer processes, with a corresponding lesser quality determination of the appropriate gain factor; additional processes may also be used. The contrast control 110 determines a maximum gain g1 that can be used without introducing contrast anomalies. That is, if the convolution value C is positive, and the Yin values correspond to a relatively dark area (low Yin values), a large enhancement could produce high-contrasting white values (high Yout values) which will appear as a white sparkles. Similarly, if the convolution value C is negative, and the Yin values correspond to a light area (high Yin values), a large negative enhancement could appear as black speckles (low Yout values). For example, a commonly used range of pixel values is 0 to 255, thereby allowing the pixel value to be processed as a byte, and a commonly used maximum gain factor g1 in a contrast control 110 is:
g1=Yin/255 if C greater than 0xe2x80x83xe2x80x83(2a)
g1=(255xe2x88x92Yin)/255 if C less than =0xe2x80x83xe2x80x83(2b).
The dynamic range control 120 determines a similar maximum gain factor g2, to suppress exaggerated overshoots, and the adaptive coring 130 determines a maximum gain factor g3, to enhance noise reduction. For example, in a high noise environment, the maximum gain g3 is kept low if the convolution value C is small, to prevent noise induced changes from being accentuated, while allowing large changes, corresponding to edges in the image, to receive a larger gain.
The output Yout from the adder 180 will be clipped to lie within the minimum and maximum range of pixel values. The clipping prevention element 140 determines a maximum gain factor g4 to minimize the aliasing that is produced by excessive output clipping. In an example embodiment, the image is divided into blocks, and the number of clippings that occur within each block is used to determine a maximum gain factor associated with each block that will have the effect of reducing the number of clippings within the block. This block-level information is transferred to the pixels by computing a gain factor g4 for each pixel, based on a bilinear interpolation of the block gain factors. To avoid rapid changes in the time domain, the block gain factors are low pass filtered, using for example a weighted sum of prior gains.
Each of these maximum gain factors g1, g2, g3, g4 are determined substantially heuristically, each based on a particular set of criteria, and often produce substantially different results. For example, a large Yin value and positive convolution value C will produce a relatively high gain g1 (equation 2a), but, a large Yin value will often result in a relatively small gain g4, to minimize clipping. These potentially conflicting gains are reconciled by a conservative selection process: the gain selector 160 selects the minimum of the gains g1, g2, g3, and g4 as the appropriate gain to be used. That is, the minimum gain among the maximum determined gains g1, g2, g3, g4 is selected in order to provide a maximum sharpness enhancement while attempting to avoid the possible effects of over-enhancing the pixel values.
The above described prior art processes are computationally complex, and typically require the use of data derived from a prior image to determine the gains to be used on a current image. The results of these processes can produce anomalous results if fast motion is present between one image and the next. Also, the prior art processes require communication between the processes that can lead to a high bandwidth requirement, particularly when some processes are implemented in software and other processes are implemented in hardware. In particular, the convolution kernel 150 preferably resides in hardware, because of the uniform and repetitive nature of the convolution algorithm, whereas the clipping prevention module 140 preferably resides in software, because of the heuristic rules-based algorithm typically employed. Often, the preferred partitioning of processes cannot be achieved, due to bandwidth limitations. In a multimedia processing system, for example, a preferred partitioning based on the appropriate embodiment to effect each particular task, or to facilitate parallel processing, often cannot be realized, due to the processor-bus bandwidth requirements that such a partitioning would produce. Often, tasks which should be performed in software are embodied in hardware, and vice versa, so as to minimize the amount of data that is transferred via the processor-bus.
It is an object of this invention to provide a less complex method of clipping prevention during picture sharpness enhancement. It is a further object of this invention to reduce the inter-process dependencies during picture sharpness enhancement. It is a further object of this invention to reduce the conflicting results of contrast control and clipping control during picture sharpness enhancement. It is a further object of this invention to minimize the temporal data dependencies during picture sharpness enhancement. It is a further object of this invention to dynamically adjust the picture sharpness enhancement based on the overall enhancement potential of each picture and the overall noise associated with the channel.
These objects and others are achieved by providing a contrast control and clipping device that is configured to determine a margin of enhancement that may be applied to each pixel. This margin is used to limit the enhancement that is determined by the conventional convolution kernel and gain control modules. Preferably, the margin is independent of the determined convolution value, thereby minimizing the bandwidth requirements among the modules used for picture sharpness enhancement. The reduced bandwidth requirement allows for an efficient partitioning of tasks between hardware and software embodiments, and eases the overall system design task. In a multi-media application, the amount of data that is transferred between the processor and sub-components is substantially reduced, and the efficient partitioning facilitates parallel processing. The margin is determined based on the pixel values in the current picture, using a computationally simple algorithm, thereby minimizing anomalies caused by temporal dependencies among the modules used for picture sharpness enhancement. Additionally, the sharpness enhancement is dynamically adjusted based on overall picture-level characteristics, including the average enhancement range, overall noise measures, and so on.